Journal of Technology Education Vol. 24 No. 1, Fall 2012 

Analysis of Five Instructional Methods for 
Teaching Sketchpad to Junior High Students 

This manuscript addresses a problem teachers of computer software 
applications face today: What is an effective method for teaching new computer 
software? Technology and engineering teachers, specifically those with 
communications and other related courses that involve computer software 
applications, face this problem when teaching computer software designed to 
assist in graphic design, web design, programming, robotics, etc. The question 
of what instructional method would prove most effective is one that affects not 
only teachers but also IT trainers, as computers and computer software 
applications continue to be the primary tools of work and leisure. Despite the 
increase in computer software application use, the associated literature on 
instructional techniques used to teach computer software is inconclusive in 
regard to which instructional methods are the most appropriate for teaching new 
software, especially in junior high technology and engineering classrooms. 

This study was designed to help identify best practices for teaching a new 
computer software application to junior high students. Five commonly used 
instructional techniques by technology education teachers were used to teach a 
new computer software, Sketchpad, to various samples of junior high 
technology education students. 

Related Literature 

A review of literature related to technology teaching, software application 
instruction, instructional methodologies, and other related topics was performed 
as part of this research study. However, because the focus of this paper is to 
describe and analyze teaching a new computer software application to junior 
high students using five different instructional methods, the literature review 
will focus on a description of the five instructional techniques used in this study. 

Instructional Methods 

Throughout history, there have been many instructional methods 
documented (Egal, 2009). The literature associated with technology education 
reveals five commonly used, cited, and recommended instructional methods. 
These methods include: (1) direct instruction, (2) problem-based learning, (3) 
video-based tutorial learning, (4) cooperative/collaborative learning, and (5) 
book/written script tutorial learning. 
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Direct Instruction 

Direct instruction, a term first coined by Rosenshine (1976), is a teacher- 
centered instructional method (Schuman, 1998) that typically follows a process 
in which teachers present new information followed by classroom activities that 
incorporate structured, guided, and independent student practice. While many 
research studies have found direct instruction to be an effective instructional 
strategy (Bock, Stebbins, & Proper, 1977), the recent push towards hands-on, 
student-centered curricular activities has resulted in direct instruction becoming 
less popular with many teachers (Magliaro, Lockee, & Burton, 2005). 

Problem-based Learning 

Problem-based learning is an instructional strategy in which problems form 
the organizing focus and stimulus for student learning. Distinguishing features 
of problem-based learning include teachers accepting the role of facilitators and 
students assuming major responsibility for their learning as they engage in 
problem-solving activities. Students are typically presented with problems and 
then work in small, self-directed learning groups to investigate and develop 
solutions to given problems (Barrows, 1996). While problem-based learning can 
be difficult to implement in the classroom (Liu, 2004), benefits from problem- 
based learning include development of higher-level thinking skills (Duch, 2001), 
long term content retention (Norman & Schmidt, 1992), better attitudes toward 
learning, higher motivation (Albanese, 1993; Norman & Schmidt, 1992), and the 
development of students problem solving skills (Gallagher, 1997; Hmelo & 
Ferrari, 1997). 

Video-based Tutorial Learning 

With the popularity of computer-based instruction, video-based tutorials as 
a means to learn various software programs have become commonplace as is 
evidenced with a quick YouTube search of most major software programs. The 
perceived advantage of this instruction method is that students are able to watch, 
review, and utilize lesson recordings in whatever manner best suits their 
educational needs. Some studies have reported positive findings in relation to 
video-based tutorials, reporting a greater ability for students to construct, or 
discover, their own knowledge (Bork, 2000) or that foreign students with 
weaker language skills prefer Web-based tutorials to traditional class lectures 
(Sweeney & Ingram, 2001). Flowever, Merino and Abel (2003) reported 
findings, which are consistent with other studies, that there was no statistical 
significant difference in student learning when comparing video tutorials and 
traditional lectures. 

Cooperative/Collaborative Learning 

In a cooperative or collaborative learning structure, students work in small 
groups to accomplish a task and are usually rewarded based upon the 
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performance of the group. Deutsch (1962) first conceptualized the three types of 
interpersonal goal structures that are typically used in classrooms: cooperative, 
competitive, and individualistic. These goal structures specify the type of 
interdependence that exists among students as they strive to accomplish learning 
goals. While many teachers agree that there is a time in which each of the goal 
structures should be appropriately used, research (Johnson, & Johnson, 1995) 
indicates that students participating in cooperative learning environments 
perform as well or better than students in competitive and individualistic 
learning environments on measures of achievement and attitudes toward 
learning. 

Book/written Script Tutorial Learning 

In typical text tutorials, students are expected to read the text, answer key 
questions posed to them in the text, and retain the knowledge for future use. The 
addition of images, graphs, and iconic cues has increased the effectiveness of 
textbook learning (Winn, 1987; Kamil, 2010; Guri-Rozenblit, 1988). 

Methodology 

This study included using each of the instructional techniques described 
above to teach a new computer software, Sketchpad, to a sample of technology 
education junior high students and then analyzing the impact that each technique 
had on student learning by giving them an assignment to use Sketchpad to 
design a CD cover of a band or artist of their choice. 

Students 

The students in our study were between the ages of 11 and 13 and were 
registered in a public junior high school 7 th or 8 th grade Intro to Technology 
class. Intro to Technology is part of the Utah CTE (Career and Technology 
Education) core classes that are designed to introduce students to technology 
and allow exploration of technological systems and their impacts on society 
(Utah State Board of Education, 2010). Demographic information such as grade 
point average, socioeconomic status, computer experience, and computer-based 
multimedia program experience was collected. This information was used to 
ensure that the sample size was homogeneous. 

Teachers 

Schools and teachers were selected because teachers had a similar number 
of years teaching, facilities were comparable, student demographics were 
similar, and teachers had multiple periods of the Intro to Technology course. 
Each teacher was assigned one of the methods identified in the literature review 
as the method of instruction to use when teaching the new program to the 
students. Teachers were asked to adhere strictly to their assigned instructional 
method while involved in this study. Teaching styles were assigned randomly to 
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teachers who were provided an explanation of the teaching style, definitions, 
examples, outlines, and associated procedures as a guide for their teaching 
experience. The authors recognize that there may be a reliability issue or 
limitation in assigning and expecting a teacher to properly use the assigned 
teaching style. However, video recordings of the teacher using the method were 
made and later analyzed by three education professors who verified that the 
teaching methods were appropriately implemented. Teachers were provided 
with cameras and recorded for approximately 90 minutes. Teachers positioned 
the camera such that the majority of the class was visible and teacher-student 
interactions were captured digitally. This verification process helped reduce this 
limitation. 

Software Program 

The software program to be taught needed to be new and unfamiliar to all 
student participants. Sketchpad is an online image creation and editing software 
developed by Mugtug, an online community dedicated to the development of 
free online programs for image editing and creation. Sketchpad was chosen 
because it: (a) is a program similar to other image-editing programs typical to 
the multimedia industry; (b) is advertised as easy to use; (c) has buttons, effects, 
and options similar to other multimedia programs; and (d) has a relatively small 
number of tools and options, which provided for a smaller learning curve. 
Sketchpad is a strictly online program, requiring no download, and allows for an 
easy download of the finished product upon completion. 

Prior to the study, it was confirmed that this software had never been taught 
to the student participants. The software was taught for two 90-minute class 
periods. Although some might argue this is insufficient to establish a cause and 
effect relationship between instructional methodology and the outcome, this 
time allotment was appropriate for this study, as it fit within the typical amount 
of time that each teacher reported they used for introducing software. For 
example, in the reporting of the demographic information, each teacher reported 
that they usually spend 1-2 class periods (60 minutes each) introducing the 
basics of a software. Concerning this, one teacher clarified, “I usually spend 
only 1-2 classes teaching the students the basics of the new tool (ex. Adobe 
Illustrator), and then in following classes, students work on their projects using 
the tool. 1 find a brief intro suffices for my students.” 

Data Collection 

Data was collected in multiple ways: (1) students and teachers completed a 
survey regarding their perceptions of the effectiveness of different types of 
instructional methods, (2) teachers were video recorded while teaching the 
software (Sketchpad) using the assigned method of instruction, (3) students 
created a CD cover for an artist or band of their choice using the program taught 
in class, and (4) student work was graded by a panel of 20 graders. The panel 
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consisted of students and professor from a college-level graphic design course. 
The average grade of each product was used as the reported data point. 

Surveys 

Students completed a Likert-based survey prior to creating the CD cover. 
The survey questions included items such as: How much experience do you 
have with multimedia programs on the computer? How familiar are you with 
computers? 

One key element of the self-report survey was asking students what they 
believe is the most effective method for teaching a new computer software 
application. Students were given options and definitions for each method of 
instruction. Students were asked to differentiate effectiveness of teaching 
methods for themselves and for their classmates. Students were also asked to 
identify the method that they perceived the teacher in their class used most 
frequently. 

Teachers also completed a survey prior to teaching the students. The teacher 
survey consisted of 20 questions about issues regarding teaching experience, 
class size, technology equipment use, teaching style, education, and multimedia 
program experience. Teachers were asked to answer each question while 
considering only the specific class the study was being conducted in. These 
responses were analyzed to ensure that teachers were similar and that each 
teacher had a broad base of technology education experience to draw from. 

Teachers were also asked to identify personal tendencies, preferences, and 
effectiveness in using different methods for their classroom. Teacher responses 
were compared with student responses to determine what relationship teacher- 
student perceptions have in regard to instructional methods. 

The teacher survey also contained questions relating to their students’ grade 
point average, socioeconomic status, computer experience, multimedia program 
experience, and average class assignment grade. These results were cross- 
analyzed with similar questions posed to students to verify data validity and 
reliability. These results also helped ensure that items such as student computer 
experience and average grade on assignments were comparable for different 
classrooms involved in the study. 

Classroom Instruction 

Each teacher was assigned an instructional method to teach Sketchpad. 
Teachers were provided with a definition of the method of instruction and asked 
to strictly adhere to this method of instruction. Teachers introduced the 
assignment to students, introduced the associated rubric, and gave the students a 
timeline for completion. Teachers were given a copy of the rubric outlining how 
the final CD covers were to be graded. Teachers then taught Sketchpad to the 
students. Each teacher completed the study during the course of two class 
periods (90 minutes), while video recording the instruction. 


- 58 - 



Journal of Technology Education Vol. 24 No. 1, Fall 2012 


Teachers video recorded themselves while teaching. The video recordings 
were watched by three education professors who used a specific rubric to ensure 
that the assigned teaching techniques were indeed the actual technique used. The 
professors unanimously reported that there were no deviations. 

Student Assignment 

Students were given the grading rubric and description of the assignment 
before working on the computer. Then students were taught SketchPad before 
they did the assignment. Students were given 60 minutes to produce the CD 
cover either by themselves or in a group, depending on the assigned method of 
instruction. As part of the study, students were informed that their participation 
in the survey and study would have no impact on their grade. Student work was 
graded at a later date according to the provided rubric by a panel of graders with 
design background. 

Grading 

Twenty students and a professor from a college-level design course graded 
the student work. Graders received a copy of the rubric and assignment 
instructions to assist them in grading. Each student-produced CD cover was 
assigned a grade on a Likert scale from 1-5 by each of the graders. Graders were 
blind to the student name, class, and instructional method. Student scores were 
compiled from each grader, and an average score for each student and then each 
class was obtained. The average grade received by students from each class was 
compared with the instructional method used in that class in an attempt to 
identify effectiveness of each method. 

Data Analysis 

Student demographic information was analyzed to ensure similar 
populations, similar familiarity with technology and computers, and similar 
experience in multimedia classroom settings. The average scores for student 
work in each class was collected and compared with the method of instruction 
provided, resulting in an average score for each method of instruction. 
Additionally, surveys for teachers and students were collected and cross- 
analyzed. The student’s perceptions of methods used in the classroom were 
compared with the methods identified by the teachers in an effort to identify 
similarities and disparities in perceptions of instructional methods. Data was 
aggregated for statistical analysis. Two specific measures of significance were 
performed with regards to the data—a t-test and an effect-size test. Although the 
authors believe the t-test and effect-size test were appropriate for this study, they 
recognize that statistical power is directly related to sample size. Because the 
sample of this study was limited to 87 student participants, the authors believe 
the findings are limited to helping create only an understanding regarding 
teaching Sketchpad to 7-8 th grade junior high technology education students, but 
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is not telling for a larger population. Consequently, additional research should 
be done to further corroborate these findings. 

Findings 

The most prevalent findings of this study are: (a) teachers and students have 
different perceptions about the effectiveness of different instructional 
techniques, (b) teachers and students have different perceptions regarding 
frequency of use of instructional methods in class, and (c) student perceptions of 
higher instructional effectiveness did not correlate with higher grades received 
for the assignment. 

Student and Teacher Perceptions about Effectiveness of Instructional 
Techniques 

There is a disconnect between what teachers and students perceive as 
effective instructional techniques: (1) students perceive book learning to be the 
most effective method of instruction for themselves and their classmates, and (2) 
teachers perceived direct instruction to be the most effective method of 
instruction and book learning to be the least effective method of instruction. 

Student Perceptions 

Students perceive book learning to be the most effective method of 
instruction for themselves and their classmates, ranking book learning above all 
other forms of learning in effectiveness for their classmates’ learning (Table 1). 
The variance between responses showed statistical significance (t = 2.57, 4.01, 
4.06, 3.6). 

Table 1 

Student Ranking of Effectiveness of Instructional Methods for Their Classmates ’ 
Learning 


Instructional Method 

Mean Score 

Book/Written Script Tutorial Learning 

3.04 

Problem-Based Learning 

2.76 

Direct Instruction 

2.63 

Collaborative Learning 

2.57 

Video-Based Tutorial Learning 

2.55 


The difference between the two highest ranked methods (book learning and 
problem-based learning) was .28 (3.04-2.76), suggesting statistical significance 
(t = 2.57). This means that students not only perceive book learning as most 
effective for their classmates but the gap between book learning and the next 
most effective method (problem-based learning) is significant—suggesting an 
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important difference for respondents between the effectiveness of each method 
of instruction (Table 2). 

Table 2 

Statistical Analysis of Student Ranking of Instructional Methods for Their 
Classmates ’ Learning 


Data Sets 

Compared 

Mean 

Std. Deviation 

t 

d 

r 

r 2 

Book/Problem- 

Based 

3.04; 2.76 

1.285; 1.02 

2.57 

.24 

.12 

.014 

Book/Video 

3.04; 2.55 

1.285; 1.33 

4.01 

.37 

.18 

.032 

Book/Collaborative 

3.04; 2.57 

1.285; 1.19 

4.06 

.38 

.19 

.032 

Book/Direct 

3.04; 2.63 

1.285; 1.148 

3.6 

.34 

.17 

.029 

Problem- 

Based/Direct 

2.76; 2.63 

1.02; 1.148 

1.28 

.12 

.06 

.003 

Problem- 

Based/Video 

2.76; 2.55 

1.02; 1.33 

1.90 

.18 

.09 

.008 

Problem- 

Based/Collaborative 

2.76; 2.57 

1.02; 1.19 

1.83 

.17 

.09 

.008 

Video/Direct 

2.55; 2.63 

1.33; 1.148 

.69 

.06 

.03 

.001 

Video/Collaborative 

2.55; 2.57 

1.33; 1.19 

.17 

.02 

.01 

.0002 

Direct/Collaborative 

2.63; 2.57 

1.148; 1.19 

.55 

.05 

.03 

.001 


When book learning was compared with each of the other identified 
teaching methods, it was the only method to show statistical significance in the 
average mean difference in every comparison (i.e., book learning compared with 
video tutorial learning, book learning compared with direct instruction, and so 
forth). No other method had such statistical significance. 

Several possible reasons could be cited for this perception. First, books 
often include images, graphs, screenshots, step-by-step instructions, and other 
tools that may assist the learning of a new computer software application. 
Although video tutorials can provide similar media content, books allow 
students the ability to tangibly hold the instructional material and go at their own 
pace of learning. A book can be easily consulted for questions and can help the 
reader to access needed information quickly and repeatedly if needed (Kamil, 
2010 ). 

Second, it is possible that student perception is skewed by the common 
practice of book learning, and students simply assume that book learning is the 
best way because that’s what they perceive most of their teachers use. Up 
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through and including junior high, textbooks are the “primary mediator of 
learning” for students in and outside of the classroom (Kamil, 2010). 

The third possibility for this finding is that, developmentally, junior high 
students are not quite ready to be self-learners (where they no longer need as 
much teacher-led learning). In Perry’s (1970) theory of intellectual and moral 
development, Perry states that students begin their development “trusting 
authority figures” at a young age, but they later seek to know the “right answer” 
on their own as they mature. At the junior high level, students are still in the 
very beginning stages of intellectual and moral development, which may be the 
reason students perceive book learning as so effective—it’s a built-in authority 
figure that they can reference whenever needing to find the "right answer.” 

Students were also asked to identify the effectiveness of instructional 
methods for their own learning. Although learning styles were not taken into 
account in this research study, this question did allow students to independently 
identify which method(s) of instruction they believe is (are) most effective for 
their own learning. Students were not instructed to think about any one 
particular class or subject in reference to this question. 

Students reported that they believed book learning is the most effective 
method of instruction for their own learning (Table 3). Similar to the previous 
question, students were not asked what method of instruction they preferred, but 
rather what method of instruction they perceive as most effective for their own 
learning. The difference in average scores of effectiveness for book learning 
when compared with each other method was statistically significant (t = 2.64, 
4.54, 3.17, 2.93). Additionally, when compared for educational significance 
(Table 4, next page) each variance for book learning compared to other forms of 
learning showed educational significance (d = .25, .43, .3, .27). 

Table 3 

Student Ranking of Effectiveness of Instructional Methods for Their Own 
Learning 


Instructional Method 

Mean Score 

Book/Written Script Tutorial Learning 

3.02 

Problem-Based Learning 

2.71 

Direct Instruction 

2.66 

Collaborative Learning 

2.63 

Video-Based Tutorial Learning 

2.45 
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Table 4 

Statistical Analysis of Student Ranking of Effectiveness of Instructional Methods 
for Their Own Learning 

Data Sets Compared 

Mean 

Std. Deviation 

t 

d 

r 

r 2 

Book/Problem-Based 

3.02; 2.71 

1.34; 1.16 

2.644 

.25 

.12 

.014 

Book/Video 

3.02; 2.45 

1.34; 1.34 

4.54 

.43 

.21 

.044 

Book/Collaborative 

3.02; 2.63 

1.34; 1.29 

3.17 

.3 

.15 

.02 

Book/Direct 

3.02; 2.66 

1.34; 1.29 

2.93 

.27 

.14 

.02 

Problem-Based/Video 

2.71; 2.45 

1.34; 1.34 

2.07 

.19 

.1 

.01 

Problem-Based/Direct 

2.71; 2.66 

1.34; 1.29 

.41 

.04 

.02 

.0004 

Problem- 

Based/Collaborative 

2.71; 2.63 

1.34; 1.29 

.651 

.06 

.03 

.001 

Video/Direct 

2.45; 2.66 

1.34; 1.29 

1.71 

.16 

.08 

.006 

Video/Collaborative 

2.45; 2.63 

1.34; 1.29 

1.46 

.14 

.07 

.005 

Collaborative/Direct 

2.63; 2.66 

1.34; 1.29 

.244 

.02 

.01 

.0001 


This is an important finding because in the high-tech, fast-paced, and 
increasingly digital world, students still perceive book learning as more effective 
than learning from a video tutorial. The availability of video tutorials and online 
videos in general has increased dramatically in the past 10 years (Tew, 2007), 
but, despite the increased availability, students in this study ranked book 
learning as more effective than video tutorials. Not only did students rank book 
learning as more effective than video tutorials but students ranked video 
tutorials as the least effective method of instruction. 

Although students believe working alone in a book based environment for 
the purposes of learning a new software application is most effective, students 
do not appear to think working in groups is completely ineffective. The data 
suggests that group work (collaborative learning) is considered effective as long 
as they are working with a common problem (problem-based learning) in mind. 

It is equally important to note that students in this study ranked the 
effectiveness of instructional methods for themselves in the exact same order as 
they reported for their classmates. Although no learning style preferences were 
considered in this study, the data suggests that students perceive personal and 
peer learning styles to be similar. 

Teacher Perceptions 

Teachers perceived direct instruction to be the most effective method of 
instruction and book learning to be the least effective method of instruction. In 
addition to student perceptions regarding most effective learning methods. 
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teacher’s perceptions were recorded and analyzed. Teachers were asked to rate 
the identified methods according to their perceived level of effectiveness in their 
class. Teachers used a 5-point Likert-type scale when ranking each method of 
instruction from 1 ( not effective) to 5 (very effective). 

The findings reveal that teachers believe direct instruction is superior to the 
other methods of instruction; not surprisingly, the teachers also reported that 
they most commonly use direct instruction in class (see table 5). 

Table 5 

Teacher Ranking of Effectiveness of Instructional Methods for Student Learning 


Method of Instruction 

Mean Score 

Book/Written Script Tutorial Learning 

2 

Problem-Based Learning 

2.6 

Collaborative Learning 

2.6 

Video-Based Tutorial Learning 

2.8 

Direct Instruction 

4.6 


Converse to what students reported to be the most effective instructional 
style, teachers believed that book learning is the least effective method of 
instruction for students. The difference in mean score for direct instruction when 
compared with other forms of instruction (Table 6, next page) returned a r-tcst 
value of 5.09, 4.27, 3.53, and 2.55—suggesting a statistically significant teacher 
preference towards direct instruction. The effect size for each comparison was 
likewise significant (d= 3.22, 2.7, 2.23, 1.61). 
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Table 6 

Statistical Analysis of Teacher Ranking of Effectiveness of Instructional 

Methods for Studen t Learning 

Data Sets Compared 

Mean 

Std. Deviation 

t 

d 

r 

r 2 

Direct/Book 

4.6; 2 

.55; 1 

5.09 

3.22 

.85 

.72 

Direct/Problem-Based 

4.6; 2.6 

.55; .89 

4.27 

2.7 

.8 

.64 

Direct/Collaborative 

4.6; 2.6 

.55; 1.14 

3.53 

2.23 

.75 

.56 

Direct/Video 

4.6; 2.8 

.55; 1.48 

2.55 

1.61 

.63 

.4 

Collaborative/Book 

2.6; 2 

1.14; 1 

.88 

.56 

.27 

.08 

Collaborative/Video 

2.6; 2.8 

1.14; 1.48 

.24 

.15 

.08 

.01 

Collaborative/Problem- 

2.6; 2.6 

1.14; .89 

0 

0 

0 

0 

Based 







Book/Video 

2; 2.8 

1; 1.48 

1 

.63 

.3 

.09 

Book/Problem-Based 

2; 2.6 

1; .89 

1 

.4 

.2 

.04 

Video/Problem-Based 

2.8; 2.6 

1.48; .89 

.26 

.16 

.1 

.01 


Student and Teacher Perceptions about Instructional Methods Used in the 
Classroom 

A comparison was performed of student perceptions of instructional 
methods used in class and teacher perceptions of instructional methods used in 
class. Two themes were discovered: (1) students perceived book learning to be 
the most commonly used method of instruction used in class and direct 
instruction to be the least commonly used method; (2) conversely, teachers 
reported using direct instruction the most and book learning the least. This 
finding is interesting because it shows a disconnect between student and teacher 
perceptions. Each of these issues is discussed below. 

Student Perceptions 

Students perceived book/written script tutorial learning to be the most 
commonly used instructional method in class (Table 7, next page). Strangely, 
students perceived direct instruction, which provided the highest grades for 
students, to be the least commonly used method of instruction. 
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Table 7 

Statistical Analysis of Student Ranking of Frequency of Use of Different 
Instructional Methods in Class 


Data Sets Compared 

Mean 

Std. Deviation 

t 

d 

r 

r 2 

Book/Problem-Based 

3.078; 

2.939 

1.401; 1.156 

1.158 

.11 

.054 

.003 

Book/Direct 

3.078; 

2.52 

1.401; 1.315 

4.395 

.411 

.2 

.04 

Book/Collaborative 

3.078; 

2.86 

1.401; 1.33 

1.71 

.16 

.08 

.01 

Book/Video 

3.078; 

2.73 

1.401; 1.33 

2.73 

.254 

.13 

.02 

Problem-Based/Direct 

2.939; 

2.52 

1.156; 1.315 

3.62 

.34 

.17 

.03 

Problem-Based/Video 

2.939; 

2.73 

1.156; 1.33 

1.79 

.17 

.08 

.01 

Problem- 

Based/Collaborative 

2.939; 

2.86 

1.156; 1.33 

.68 

.06 

.03 

.001 

Video/Direct 

2.73; 

2.52 

1.33; 1.315 

1.7 

.16 

.08 

.01 

Video/Collaborative 

2.73; 

2.86 

1.33; 1.33 

1.05 

.1 

.05 

.003 

Collaborative/Direct 

2.86; 

2.52 

1.33; 1.315 

2.75 

.26 

.13 

.02 


Students perceived teachers as using book learning more than any other 
method of instruction in class (t = 1.158, 4.39, 1.71, 2.73) and much more than 
direct instruction (t = 4.39). Also, students perceived their teachers as using 
books to teach materials far more frequently than videos or other multimedia. 

Teacher Perceptions 

While students reported book learning to be the most commonly used 
method of instruction in class and direct instruction to be the least commonly 
used method, teachers reported the opposite—reporting using direct instruction 
far more than any other method of instruction (see table 8, next page). Teachers 
ranked book learning, which was ranked by the students to be the most used 
technique, to be the least used method. 
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Table 8 

Teacher Ranking of Frequency of Use of Different Instructional Methods in 
Class 


Method of Instruction 

Mean Score 

Book/Written Script Tutorial Learning 

2.4 

Problem-Based Learning 

2.4 

Collaborative Learning 

2.8 

Video-Based Tutorial Learning 

2.4 

Direct Instruction 

4.6 


Instead, the teachers ranked direct instruction as being used significantly 
more than any other method (4.6 average rating compared with 2.8 for 
collaborative learning, which was ranked second). When compared with the 
other methods (Table 9) of instruction the variance was statistically significant 
in each comparison (t = 4.7, 4.7, 3.29, 2.8). When compared for an effect size, 
educational significance was also found in each scenario (d= .83, .83, .72, .66). 
The difference in student and teacher perceptions is alarming when considering 
that students and teachers both show strong leanings about which method of 
instruction is most effective. 

Table 9 

Statistical Analysis of Teacher Ranking of Frequency of Use of Different 
Instructional Methods in Class 


Data Sets Compared 

Mean 

Std. Deviation 

t 

d 

r 

r 2 

Direct/Book 

4.6; 2.4 

.55; .89 

4.1 

2.97 

.83 

.69 

Direct/Problem-Based 

4.6; 2.4 

.55; .89 

4.7 

2.97 

.83 

.69 

Direct/Collaborative 

4.6; 2.8 

.55; 1.09 

3.29 

2.09 

.72 

.52 

Direct/Video 

4.6; 2.4 

.55; 1.67 

2.8 

1.77 

.66 

.44 

Collaborative/Book 

2.8; 2.4 

1.09; .89 

.64 

.4 

.2 

.04 

Collaborative/Video 

2.8; 2.4 

1.09; 1.67 

.45 

.28 

.14 

.02 

Collaborative/Problem- 

Based 

2.8; 2.4 

1.09; .89 

.64 

.4 

.2 

.04 

Book/Video 

2.4; 2.4 

.89; 1.67 

0 

0 

0 

0 

Book/Problem-Based 

2.4; 2.4 

.89; .89 

0 

0 

0 

0 

Video/Problem-Based 

2.4; 2.4 

1.67; .89 

0 

0 

0 

0 


Effectiveness of Instructional Methods in Teaching Sketchpad 

Each student produced a CD cover using the software application taught in 
class. Students were given approximately 60 minutes to create their CD cover 
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and turn it in electronically (85 CD covers were graded in total). A panel of 20 
graders with design background graded the student work. Graders were blind as 
to the method of instruction received and graded student work on a 1-5 Likert 
scale. A grading rubric was provided to the graders. 

Student grades for each group were combined and a class average grade was 
obtained (Table 10). Each class average was compared and analyzed to 
determine how effective each method of instruction proved to be in respect to 
the grade given. Student perceptions of higher instructional effectiveness did not 
correlate with higher grades received for the assignment. In fact, the data show 
that students receiving direct instruction scored higher than any other method of 
instruction. When compared with other methods of instruction (Table 11, next 
page) a significant difference in variance between scores for students receiving 
direct instruction and those receiving other instructional methods was shown for 
multiple comparisons (t = 2.65, .45, 2.63, .95). 

Table 10 

Average Grade Received by Students—Separated by Instructional Method Used 


Instructional Method Received 

Average Grade 

Direct Instruction 

3.02 

Problem-Based Learning 

2.95 

Book/Written Script Tutorial Learning 

2.87 

Video-Based Tutorial Learning 

2.49 

Collaborative Learning 

2.43 
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Table 11 

Statistical Analysis of Average Grade Received by Students—Separated by 
Instructional Method Used 


Data Sets 

Compared 

Mean 

Std. Deviation 

t 

d 

r 

r 2 

Direct/Collaborative 

3; 2.43 

.48; .52 

2.654 

1.13 

.49 

.232 

Direct/Problem- 

Based 

3; 2.95 

.48; .32 

.447 

.122 

.06 

.002 

Direct/Video 

3; 2.48 

.48; .70 

2.63 

.866 

.40 

.016 

Direct/Book-written 

3; 2.86 

.48; .24 

.9478 

.369 

.18 

.032 

Book/Collaborative 

2.86; 2.43 

.24; .53 

2.41 

1.05 

.46 

.211 

Book/Problem- 

Based 

2.86; 2.95 

.24, .32 

.774 

.32 

.16 

.025 

Book/Video 

2.86; 2.48 

.24; .70 

1.71 

.73 

.34 

.116 

Video/Collaborative 

2.48; 2.43 

.70; .53 

.195 

.08 

.04 

.002 

Video/Problem- 

Based 

2.48; 2.95 

.70; .32 

2.63 

.86 

.40 

.16 

Problem- 

Based/Collaborative 

2.95; 2.43 

.32; .53 

3.08 

1.19 

.51 

.26 


The combined validity of multiple tests (/-test, Cohen’s d) adds weight to 
the assertion that direct instruction appears to be more effective than 
collaborative learning or video-based tutorials in helping students score higher 
when taught a new computer software application at the junior high level. In 
summary, despite teacher and student perceptions regarding effectiveness and 
frequency of use of different instructional methods, direct instruction proved to 
produce the best grades for students when taught a new computer software 
application. 


Conclusions and Recommendations 

Based on the findings from this study, several conclusions and 
recommendations can be generalized for application by teachers of computer 
software applications like Sketchpad. Three are discussed below. 

Use of Book Learning at the Junior High Level 

Teachers need to involve the use of book learning—especially at the junior 
high age level. At the junior high level, students are still in the early stages of 
their own intellectual development (Perry, 1970), and students want (or are used 
to) an authority for everything they do. Students want to have someone tell them 
the “right way” of doing each thing and the “right answer” for each question 
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they encounter (Perry, 1970). A book is also another authority figure in the 
classroom—the book can be a source of “right answers” and “right ways” for 
students when the teacher is not available. Books can provide a constant stream 
of hints, tips, tricks, and steps for students to follow as they learn new software 
programs. Because students can take books home, students can use them to leam 
on their own time, at their own pace, and in any desired location. 

Another aspect of learning that is critical to students at the junior high level 
is praise and positive feedback. At the junior high level, as at all age levels, there 
is a need for reinforcement and praise—often this praise and reinforcement 
comes as a confirmation that one is doing the right thing, following the steps 
correctly, and has achieved a short-term goal along the way. When student 
methods, answers, or products resemble what is outlined in the book, the student 
receives a small measure of “praise” as they reaffirmed that their learning 
corresponds with what was intended. 

It may be difficult for many teachers to institute and effectively use books 
in their classrooms; lack of books, lack of excitement for books (by the teacher 
or the students), and other factors may make book learning difficult in some 
settings. A possible alternative to a textbook is a packet for each assignment. A 
packet of instructions could be copied for each student and used as a reference 
for students to refer to throughout instruction and the process of learning. 

Understanding Student Perception of Classroom Teaching Practices 

Teachers need to understand the perceptions of their students in regard to 
the teaching practices used in the classroom. Teachers must consciously and 
consistently evaluate their own teaching practices and seek to understand the 
perceptions of their students. An understanding of student perceptions will help 
inform teachers regarding their instructional effectiveness and teaching methods 
used (Hicks, 2010). As shown in this study, frequently teacher perceptions of 
instructional methods being used do not match with methods perceived by 
students. 

Teachers should explicitly ask their students about techniques used in class 
to discover student perceptions and not rely solely on self-evaluation techniques 
for discovering effectiveness of instructional methods. Video recordings and 
post-teaching analysis (Wright, 2008) have been shown as effective in 
improving teacher cognition of methods used and improving teaching 
effectiveness. A simple survey, questionnaire, or even an open discussion with 
students could also provide such feedback for a teacher. 

Improving Direct Instruction Techniques 

In this study, direct instruction provided the highest average student grade 
for the assignment and was reported by teachers to be the most effective 
instructional method. Teachers also reported using direct instruction 
significantly more than any other method of instruction. Conversely, students 
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perceived direct instruction to be the least used method of instruction in class. 
Students also ranked book learning and problem-based learning as more 
effective than direct instruction for their own learning and their classmates 
learning. Teachers must find ways to improve the perception of direct 
instruction in the eyes of students by improving their own direct instruction 
techniques. 
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